Bayesian Frequentist Multiple Testing
نویسندگان
چکیده
We introduce a Bayesian approach to multiple testing. The method is an extension of the false discovery rate (FDR) method due to Benjamini and Hochberg (1995). We also examine the empirical Bayes approach to simultaneous inference proposed by Efron, Tibshirani, Storey and Tusher (2001). We show that, in contrast to the single hypothesis case – where Bayes and frequentist tests do not agree even asymptotically – in the multiple testing case we do have asymptotic agreement.
منابع مشابه
Bayesian and Frequentist Multiple Testing
We introduce a Bayesian approach to multiple testing. The method is an extension of the false discovery rate (FDR) method due to Benjamini and Hochberg (1995). We also examine the empirical Bayes approach to simultaneous inference proposed by Efron, Tibshirani, Storey and Tusher (2001). We show that, in contrast to the single hypothesis case – where Bayes and frequentist tests do not agree even...
متن کاملBlending Bayesian and frequentist methods according to the precision of prior information with applications to hypothesis testing
The following zero-sum game between nature and a statistician blends Bayesian methods with frequentist methods such as p-values and con dence intervals. Nature chooses a posterior distribution consistent with a set of possible priors. At the same time, the statistician selects a parameter distribution for inference with the goal of maximizing the minimum Kullback-Leibler information gained over...
متن کاملBayesian point null hypothesis testing via the posterior likelihood ratio
Neyman-Pearson or frequentist inference and Bayes inference are most clearly differentiated by their approaches to point null hypothesis testing. With very large samples, the frequentist and Bayesian conclusions from a classical test of significance for a point null hypothesis can be contradictory, with a small frequentist P -value casting serious doubt on the null hypothesis, but a large Bayes...
متن کاملObjective Bayesian Two Sample Hypothesis Testing for Online Controlled Experiments
As A/B testing gains wider adoption in the industry, more people begin to realize the limitations of the traditional frequentist null hypothesis statistical testing (NHST). The large number of search results for the query “Bayesian A/B testing” shows just how much the interest in the Bayesian perspective is growing. In recent years there are also voices arguing that Bayesian A/B testing should ...
متن کاملUnified Conditional Frequentist and Bayesian Testing of Composite Hypotheses
Testing of a composite null hypothesis versus a composite alternative is considered when both have a related invariance structure. The goal is to develop conditional frequentist tests that allow the reporting of data-dependent error probabilities, error probabilities that have a strict frequentist interpretation and that reflect the actual amount of evidence in the data. The resulting tests are...
متن کامل